Method for editing an image

ABSTRACT

Editing an image includes an image input unit reading displaying data of the image. The displaying data includes angular profiles of an object in the image, and the angular profiles include color information of the object corresponding to a plurality of viewing angles. Editing the image also includes a processor altering the angular profiles of the object to generate edited displaying data according to at least one editing instruction, and an image output unit outputting the edited displaying data.

CROSS REFERENCE TO RELATED APPLICATION

This non-provisional application claims priority of U.S. provisionalapplication No. 62/958,319, filed on Jan. 8, 2020, included herein byreference in its entirety.

BACKGROUND OF THE DISCLOSURE 1. Field of the Disclosure

The present invention is related to a method for editing an image, andmore particularly, to a method for editing angular profiles of an image.

2. Description of the Prior Art

Since the display devices have been adopted in more and more fields, therequirement for better visual effects is raised. For example, highdynamic range (HDR) displays have been developed to show high contrastimages so the details in both the bright portion and the dark portion ofan image can be seen. Although the HDR display is able to show imageswith greater brightness contrast and delivers better visual effects thanthe traditional display apparatus, the HDR display still has difficultyin showing the real light shining effects or the kirameki effect.

For example, some objects, such as butterfly wings and bubbles, canscatter the light and produce structural colors. In this case, topresent such an effect, the display device may have to display differentlighting profiles to different directions, so the people may seedifferent colors and/or light intensities when looking at the objectfrom different positions. However, the conventional display can onlyshow the fixed reflection profile of a static scene. Furthermore, evenwith the display device capable of showing the kirameki effects, thelighting profiles of the images may still need to be enhanced or editedfor providing satisfying visual quality.

SUMMARY OF THE DISCLOSURE

One embodiment of the present disclosure discloses a method for editingan image.

The method for editing an image includes an image input unit readingdisplaying data of the image. The displaying data includes angularprofiles of an object in the image, and the angular profiles includecolor information of the object corresponding to a plurality of viewingangles. The method also includes a processor altering the angularprofiles of the object to generate edited displaying data according toat least one editing instruction, and an image output unit outputtingthe edited displaying data.

Another embodiment of the present disclosure discloses an image editingsystem. The image editing system includes an image input unit, aprocessor, and an image output unit. The image input unit readsdisplaying data of an image. The displaying data includes angularprofiles of an object in the image, and the angular profiles includecolor information of the object corresponding to a plurality of viewingangles. The processor receives the displaying data from the image inputunit, and alters the angular profiles of the object to generate editeddisplaying data according to at least one editing instruction. The imageoutput unit outputs the edited displaying data.

These and other objectives of the present disclosure will no doubtbecome obvious to those of ordinary skill in the art after reading thefollowing detailed description of the embodiment that is illustrated inthe various figures and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an image editing system according to one embodiment.

FIG. 2 shows the displaying data of the image according to oneembodiment.

FIG. 3 shows the angular profile of a point on the object in the image.

FIG. 4 shows the displaying data according to one embodiment.

FIG. 5 shows a discrete cosine transform of the angular profile in FIG.3 and the displaying data.

FIG. 6 shows an original angular profile and the transformed angularprofiles.

FIG. 7 shows the angular profiles of the objects in the image and thesynthesized angular profiles.

FIG. 8 shows an image editing system according to another embodiment.

FIG. 9 shows a flowchart of a method for editing the image according toone embodiment.

DETAILED DESCRIPTION

FIG. 1 shows an image editing system 100 according to one embodiment.The image editing system 100 includes an image input unit 110, aprocessor 120, and an image output unit 130.

The image input unit 110 may read the displaying data D1 of an image forediting. In some embodiments, the displaying data D1 may include angularprofiles of objects in the image, and the angular profiles may includethe color information of the objects corresponding to a plurality ofviewing angles. The processor 120 may receive the displaying data D1from the image input unit 110, and alter the angular profiles of theobjects in the displaying data D1 to generate edited displaying data ED1according to editing instruction(s). The image output unit 130 mayoutput the edited displaying data ED1.

FIG. 2 shows displaying data D1 of the image IMG1 according to oneembodiment. In FIG. 2, the image IMG1 includes objects OB1 and OB2.Since the objects OB1 and OB2 may comprise different materials, they mayhave different angular profiles. In some other embodiments, the angularprofiles may be referred as lighting profiles. FIG. 3 shows the angularprofile of a point on the object OB1. In FIG. 3, the object OB1 may be ametal ball. Therefore, the angular profile of the point may have anintensity peak when the viewing angles are within a small region betweenA1 and A2. That is, when the viewer views the object OB1 within viewingangles A1 to A2, the viewer may see bright reflection light. However, ifthe viewer views the object OB1 outside the viewing angles A1 to A2, theviewer may not see the reflection light.

However, although the different parts of the object OB1 may comprise thesame material, the different parts of the object OB1 may still havedifferent angular profiles due to their relative positions with thelight source being different. In addition, some materials may havedifferent angular profiles for different colors. For example, when theviewer watches the bubbles or the wings of butterflies, the viewer maysee different colors when the viewer changes the viewing angle.

Therefore, in some embodiment, the displaying data D1 in FIG. 2 mayinclude a plurality of color maps. Each color map may be correspondingto a specific angle and a specific color. For example, in FIG. 2, thecolor maps RM₁ to RM_(x) may record the intensities of the red color tobe presented by the pixels of the image IMG1 for X different viewingangles, where X is an integer greater than 1.

In some embodiments, X may be, for example, but not limited to, 61. Inthis case, each of the color maps RM₁ to RM_(x) may be corresponding toone viewing angles of −30°, −29°, . . . −1°, 0°, 1°, . . . 29°, and 30°.

Similarly, the displaying data D1 may further include color maps GM₁ toGM_(x) for recording the intensities of the green color for X differentviewing angles, and color maps BM₁ to BM_(x) for recording theintensities of the blue color for X different viewing angles.

In this case, the displaying data D1 may use the red color maps RM₁ toRM_(x), the green color maps GM₁ to GM_(x) and the blue color maps BM₁to BM_(x) to record the angular profiles of the objects in the imageIMG1. However, in some other embodiments, color maps based on differenttypes of color spaces may be used. For example, the displaying data D1may include color maps for recording the luminance and chrominance basedon the YUV color space.

Furthermore, in some embodiments, to reduce the size of the displayingdata, the displaying data may record the angular profiles by fittingparameters that may describe the angular profiles. For example, theangular profile in FIG. 2 may be described by the fitting parameterssuch as the top intensity I_(top) of the angular profile, the peak angleA0 at which the angular profile has the top intensity I_(top), the baseintensity I_(base), and the dispersion factor DP as shown in FIG. 3.

FIG. 4 shows displaying data D2 according to one embodiment. In someembodiments, the displaying data D2 may be used to store the angularprofiles of the objects in the image IMG1. In FIG. 4, the displayingdata D2 may include a plurality of parameter maps PM₁ to PM_(Y), where Yis an integer greater than 1. Each of the parameter maps PM₁ to PM_(Y)may be corresponding to a specific fitting parameter of the angularprofiles. For example, the parameter map PM₁ may record the peak angles,the parameter map PM₂ may record the top intensity, and the parametermap PM₃ may record the base intensity. However, since different types ofmaterials may have different angular profiles, the fitting parametersused for describing different angular profiles may be different.Therefore, in some embodiments, the image input unit 110 may receive aheader file listing the fitting parameters used for describing theangular profiles, so the processor 120 may analyze and edit thedisplaying data D2 later.

In some embodiments, the discrete cosine transform (DCT) may be used togenerate a series of coefficients for describing the angular profile ina universal manner. FIG. 5 shows a discrete cosine transform of theangular profile in FIG. 3 and the displaying data D3. In FIG. 5, theangular profile may be transformed to the frequency domain by using thediscrete cosine transform. Since the angular profiles are usually rathersimple without too many turning points, the number of non-zero DCTcoefficients or the number of significant DCT coefficients is usuallyrather small. Therefore, in some embodiments, the displaying data D3 mayinclude a plurality DCT coefficient maps CM₁ to CM₂ corresponding to theangular profiles, where Z is an integer greater than 1. For example,each of the DCT coefficient maps CM₁ to CM₂ may be corresponding to thecoefficients of a specific frequency of the angular profiles.Consequently, the angular profiles may be recorded in a form offrequency, and the required data size for recording the angular profilesmay be reduced significantly.

After the processor 120 receives the displaying data D1 (or D2 or D3)from the image input unit 110, the processor 120 may alter the angularprofiles of the objects to generate edited displaying data ED1 accordingto editing instruction(s). In some embodiments, the image input unit 110may provide a user interface for the user to command the processor 120to perform the desired editing instructions. For example, the imageediting system 100 may present a user interface to show the image to beedited and the tools or instruction bars for editing the image. However,unlike any other software, the image editing system 100 allows editinginstructions for editing the angular profiles of the objects in theimages. The editing instructions may include a generation instruction, atransform instruction, a select instruction, a copy and pasteinstruction, and/or a synthesize instruction, but not limited thereto.

For example, when the processor 120 is requested to perform thegeneration instruction, the processor 120 may generate a new angularprofile for the object accordingly. For example, the image IMG1 mayinclude objects having constant angular profiles. That is, when theviewer changes his/her viewing angle, these objects will present thesame lighting profile, and the viewer will see the same visual effectsfrom all different viewing angles. Or the user of the image editingsystem 100 may like to change the material of the object to enhance thevisual effects of the image IMG1. In these cases, the generationinstruction may be adopted, and the processor 120 may include a databasefor storing angular profiles of different materials, such as metal,woods, and glass, so the user may select the angular profile of thedesired material and apply the angular profile to the selected object.In some embodiments, the processor 120 may need to analyze the depthinformation of the surface of the selected object, and the position ofthe light source(s), so the selected angular profile may be applied tothe object in compliance with the environment and other objects in theimage IMG1. Furthermore, in some other embodiments, the processor 120may allow the user to create a new angular profile instead of applyingthe angular profiles stored in the database directly.

In addition, when the processor 120 is requested to perform thetransform instruction, the processor 120 may sharpen the angularprofiles, broaden the angular profiles, shift the angular profiles inangles or in intensities, reverse the angular profiles in angles or inintensities, scale the angular profiles, duplicate the angular profiles,remove noises from the angular profiles, and/or add noises to theangular profiles accordingly, but not limited thereto.

For example, the processor 120 may sharpen the angular profiles byincreasing the top intensity of the angular profile and narrowing theregions of viewing angles that may see the reflection light. In thiscase, the edited displaying data ED1 will present a sharper specularreflection on the selected object. In contrary, the processor 120 maybroaden the angular profiles by decreasing the top intensity of theangular profile and widening the region of the viewing angles that maysee the reflection light. In this case, the edited displaying data ED1will present a broader specular reflection on the selected object.

In another example, the processor 120 may shift the angular profiles byshifting the angles corresponding to the peak intensities. FIG. 6 showsan original angular profile and the transformed angular profiles. InFIG. 6, the curve C0 may represent the original angular profile, and thecurve C1 may represent the angular profile being transformed by shiftingin angles. According to curve C0, the peak intensities should be seenwithin the angles A1 to A2 in the original angular profile; however,after being shifted, the peak intensities will be seen within the anglesA3 to A4 according to the curve C1. In some embodiments, shifting theangular profiles of an object in angles may create the visual effect ofrotating the object. However, in some embodiments, the processor 120 mayshift the angular profiles by shifting the top intensities and/or thebase intensities to enhance or weaken the specular reflection effect.

In addition, the processor 120 may reverse the angular profiles inangles and/or in intensities according to the transform instructions. InFIG. 6, curve C2 may represent the angular profile after being reversedin angles, and curve C3 may represent the angular profile after beingreversed in intensities.

Furthermore, the processor 120 may duplicate and scaling the angularprofile in angles. In FIG. 6, curve C4 may represent the angular profileafter being duplicated and scaled in angles. In this case, the viewermay see the specular reflection on the object from the viewing angles A5to A6 and from the viewing angles A7 to A8.

Nevertheless, the processor 120 may add noises to the angular profiles,which may make the surface of the object look bumpy. In contrary, theprocessor 120 may remove noises from the angular profiles so the surfaceof the object may look smoother.

In some embodiments, the processor 120 may alter the angular profiles ofdifferent colors according to the transformation instruction. Forexample, the processor 120 may shift the color in hue or color purity,so that the viewer may see different colors as the viewer changes theviewing angles, creating unique kirameki effects.

By providing different types of transform instructions, the imageediting system 100 would be able to create different types of effects,enriching the visual experience and enhancing the image quality.

In some embodiments, to perform the transform instruction accurately onthe selected objects, the processor 120 may need to be able to definethe boundary of the selected objects. For example, to select the object,the user may click on a spot of the object through the user interface ofthe image editing system 100. In this case, the processor 120 wouldanalyze the angular profile of the selected position of the object, thatis, the angular profile of the clicked spot, to determine the materialtype of the object. After the processor 120 has determined the materialtype of the object, the processor 120 may combine the positions adjacentto the selected position having angular profiles that are determined tobe the same material type of the object to define the boundary of theobject. For example, if the clicked position is determined to haveangular profiles similar to metal, the processor 120 may combine all theadjacent parts having the angular profiles of metal as a complete regionof the selected object. That is, the boundary of the object may bedefined by analyzing the angular profiles of the adjacent positions.

After the object is selected, the processor 120 may perform thetransformation instruction accordingly. Furthermore, in someembodiments, after the object is selected, the processor 120 may performa copy and paste instruction to copy the angular profiles of theselected object, and paste the angular profiles of the selected objectto another selected object. For example, the user may select the objectOB1 in FIG. 2, copy the angular profiles of the object OB1, select theobject OB2 in FIG. 2, and paste the angular profiles of the object OB1to the object OB2. In this case, the object OB2 will have the samemetallic angular profiles as the object OB1.

In addition, the processor 120 may synthesize at least one angularprofile to a selected object in the image according to a synthesizeinstruction. FIG. 7 shows the angular profiles of the objects OB1 andOB2 and the synthesized angular profile. In FIG. 7, the curve CA showsthe angular profile of the object OB1, the curve CB shows the angularprofile of the object OB2, and the curve CC shows the synthesizedangular profile of the object OB1 and OB2. By synthesizing the twoangular profiles, the selected area may show the shining effects of twodifferent materials at the same time, simulating the translucent visualeffects.

By using the editing instructions, such as the generation instruction,the transform instruction, the select instruction, the copy and pasteinstruction, and the synthesize instruction, the user may use imageediting system 100 to edit the displaying data D1 of the image IMG1 togenerate the edited displaying data ED1 showing the desired visualeffects.

FIG. 8 shows an image editing system 200 according to anotherembodiment. The image editing system 200 may be operated by the similarprinciples as the image editing system 100. However, the image editingsystem 200 may further include an image capturing device 240 and adisplay device 250.

The image capturing device 240 may capture images to generate thedisplaying data D1. In some embodiments, the image capturing device 240may include an multi-lens camera 242 for capturing the angular profilesof the objects. However, in some other embodiments, the image capturingdevice 240 may include a single-lens camera without using the multiplelenses if the software for constructing the angular profiles isavailable.

In addition, the image capturing device 240 may further include a depthsensor 244 and a fish eye sensor 246. The depth sensor 244 may beadopted to obtain the depth information of the objects so the angularprofiles derived by the image capturing device 240 may be even moreaccurately. The fish eye sensor 246 may be adopted to obtain thelighting map of the scene. For example, the fish eye sensor 246 mayrecord the positions of the lighting sources of the scene, making theangular profiles more realistic. However, the depth sensor 244 and thefish eye sensor 246 may be omitted according to the system requirement,that is, the depth sensor 244 and the fish eye sensor 246 may beoptional. In other embodiments, other suitable sensors may be includedin the image capturing device 240, but it is not limited thereto.

Furthermore, the display device 250 may include a display panel 252 andan optical modulator 254. The display panel 252 may be, for example butnot limited to, a liquid crystal display panel, and may present theedited displaying data ED1 by displaying a plurality of imagescorresponding to a plurality of viewing angles at the same time, and theoptical modulator 254 may direct the plurality of images displayed bythe display panel 252 to different directions. The optical modulator 254may be a lenticular lens, a liquid crystal gradient-index (GRIN) lens, aparallax barrier, a liquid crystal barrier or a light emitting diode(LED) display panel. Consequently, when the viewer watches the displaydevice 250 from different viewing angles, the viewer may see differentlighting profiles of the objects. Consequently, the result of theangular profiles edited by the image editing system 200 may be shown tothe user, allowing the user to make further adjustments to the imageIMG1 according to the displaying result.

FIG. 9 shows a flowchart of a method 300 for editing the image IMG1according to one embodiment. In some embodiments, the method 300 may beperformed with the image editing system 100 or 200. The method 300includes steps S310 to S330.

S310: the image input unit 110 reads the displaying data of the imageIMG1;

S320: the processor 120 alters the angular profiles of the objects togenerate edited displaying data ED1 according to at least one editinginstruction; and

S330: the image output unit 130 outputs the edited displaying data ED1.

In some embodiments, the image input unit 110 may read the displayingdata D1 in step S310. The displaying data D1 may include color maps RM₁to RM_(x), GM₁ to GM_(x), and BM₁ to BM_(x) to store the angularprofiles of the objects in the image IMG1. However, in some otherembodiments, the image input unit 110 may read the aforementioneddisplaying data D2 or D3 according to the system requirement. Thedisplaying data D2 may include a plurality of parameter maps PM₁ toPM_(Y), and each of the parameter maps PM₁ to PM_(Y) may becorresponding to a specific fitting parameter of the angular profiles.The displaying data D3 may include DCT coefficient maps CM₁ to CM_(z),and each of the DCT coefficient maps CM₁ to CM_(z) may be correspondingto the coefficients of a specific frequency of the angular profiles.

In step S320, the processor 120 may alter the angular profiles togenerate the edited displaying data ED1 according to different editinginstructions, such as a generation instruction, a transform instruction,a select instruction, a copy and paste instruction, and a synthesizeinstruction.

According to the generation instruction, the processor 120 may generatea new angular profile for the object. According to different types oftransform instructions, the processor 120 may sharpen the angularprofiles, broaden the angular profiles, shift the angular profiles inangles, shift the angular profiles in intensities, reverse the angularprofiles in angles, reverse the angular profiles in intensities, scalethe angular profiles, duplicate the angular profiles, remove noises fromthe angular profiles, or add noises to the angular profiles, but notlimited thereto.

Also, according to the select instruction, the processor may analyze theangular profile of a selected position of the object to determine thematerial type of the object and combine the positions adjacent to theselected position having angular profiles that are determined to be thematerial type of the object to define the boundary of the object.

After an object has been selected, the processor 120 would be able toperform some following instructions. For example, the processor 120 mayperform the copy and paste instruction. In this case, the processor 120may copy the angular profiles of the selected object and paste theangular profiles of the select object to another selected object.

In some embodiments, the processor 120 may synthesize at least oneangular profile to a selected position in the image according to asynthesize instruction as shown in FIG. 7, creating a translucent visualeffect.

In summary, the image editing system and the method for editing an imageprovided by the embodiments of the present disclosure may allow the userto edit the angular profiles of the objects in the image, therebyimproving the image quality. Furthermore, by providing different typesof transform instructions, the image editing system may help to createunique visual effects and enhance the kirameki effects for display.

Those skilled in the art will readily observe that numerousmodifications and alterations of the device and method may be made whileretaining the teachings of the disclosure. Accordingly, the abovedisclosure should be construed as limited only by the metes and boundsof the appended claims.

What is claimed is:
 1. A method for editing an image, comprising: animage input unit reading displaying data of the image, wherein thedisplaying data comprises a plurality of angular profiles of an objectin the image, and the angular profiles comprise color information of theobject corresponding to a plurality of viewing angles; a processoraltering the plurality of angular profiles of the object to generateedited displaying data according to at least one editing instruction;and an image output unit outputting the edited displaying data.
 2. Themethod of claim 1, wherein the displaying data comprises a plurality ofcolor maps, and each of the plurality of color maps is corresponding toa specific angle and a specific color.
 3. The method of claim 1, whereinthe displaying data comprises a plurality of parameter maps, and each ofthe plurality of parameter maps is corresponding to a specific fittingparameter of the plurality of angular profiles.
 4. The method of claim3, further comprising: the image input unit receiving a header filelisting a plurality of fitting parameters used for describing theplurality of angular profiles.
 5. The method of claim 1, wherein thedisplaying data comprises a plurality of discrete cosine transformcoefficient maps corresponding to the plurality of angular profiles. 6.The method of claim 1, wherein the processor altering the plurality ofangular profiles to generate the edited displaying data according to theat least one editing instruction comprises: according to a generationinstruction, the processor generating a new angular profile for theobject.
 7. The method of claim 1, wherein the processor altering theplurality of angular profile of the object to generate the editeddisplaying data according to the at least one editing instructioncomprises: according to a transform instruction, the processorsharpening the plurality of angular profiles, broadening the pluralityof angular profiles, shifting the plurality of angular profiles inangles, shifting the plurality of angular profiles in intensities,reversing the plurality of angular profiles in angles, reversing theangular profiles in intensities, scaling the plurality of angularprofiles, duplicating the plurality of angular profiles, removing noisesfrom the plurality of angular profiles, or adding noises to theplurality of angular profiles.
 8. The method of claim 1, wherein theprocessor altering the plurality of angular profile of the object togenerate the edited displaying data according to the at least oneediting instruction comprises: according to a select instruction, theprocessor: analyzing an angular profile of a selected position of theobject to determine a material type of the object; and combiningpositions adjacent to the selected position having angular profiles thatare determined to be the material type of the object to define aboundary of the object.
 9. The method of claim 1, wherein the processoraltering the plurality of angular profile of the object to generate theedited displaying data according to the at least one editing instructioncomprises: according to a copy and paste instruction, the processor:copying the plurality of angular profiles of a selected object; andpasting the plurality of angular profiles of the selected object toanother selected object.
 10. The method of claim 1, wherein theprocessor altering the plurality of angular profile of the object togenerate the edited displaying data according to the at least oneediting instruction comprises: according to a synthesize instruction,the processor synthesizing at least one angular profile to a selectedarea in the image.
 11. An image editing system, comprising: an imageinput unit configured to read displaying data of an image, wherein thedisplaying data comprises a plurality of angular profiles of an objectin the image, and the plurality of angular profiles comprise colorinformation of the object corresponding to a plurality of viewingangles; a processor configured to receive the displaying data from theimage input unit, and alter the plurality of angular profiles of theobject to generate edited displaying data according to at least oneediting instruction; and an image output unit configured to output theedited displaying data.
 12. The image editing system of claim 11,wherein the displaying data comprises a plurality of color maps, andeach of the plurality of color maps is corresponding to a specific angleand a specific color.
 13. The image editing system of claim 11, wherein:the displaying data comprises a plurality of parameter maps, and each ofthe plurality of parameter maps is corresponding to a specific fittingparameter of the plurality of angular profiles; and the image input unitis further configured to receive a header file listing a plurality offitting parameters used for describing the plurality of angularprofiles.
 14. The image editing system of claim 11, wherein thedisplaying data comprises a plurality of discrete cosine transformcoefficient maps corresponding to the plurality of angular profiles. 15.The image editing system of claim 11, wherein the processor generates anew angular profile for the object according to a generationinstruction.
 16. The image editing system of claim 11, wherein theprocessor sharpens the plurality of angular profiles, broadens theplurality of angular profiles, shifts the plurality of angular profilesin angles, shifts the plurality of angular profiles in intensities,reverses the plurality of angular profiles in angles, reverses theplurality of angular profiles in intensities, scales the plurality ofangular profiles, duplicates the plurality of angular profiles, removesnoises from the plurality of angular profiles, or adds noises to theplurality of angular profiles to according to a transform instruction.17. The image editing system of claim 11, wherein: the processoranalyzes an angular profile of a selected position of the object todetermine a material type of the object, and combines positions adjacentto the selected position having angular profiles that are determined tobe the material type of the object to define a boundary of the objectaccording to a select instruction; and the processor copies theplurality of angular profiles of a selected object, and pastes theplurality of angular profiles of the selected object to another selectedobject according to a copy and paste instruction.
 18. The image editingsystem of claim 11, wherein the processor synthesizes at least oneangular profile to a selected area in the image according to asynthesize instruction.
 19. The image editing system of claim 11,further comprising an image capturing device configured to capture theimage to generate the displaying data.
 20. The image editing system ofclaim 11, further comprising a display device comprising a display paneland an optical modulator, wherein the display panel is configured topresent the edited displaying data by displaying a plurality of imagescorresponding to a plurality of viewing angles at the same time, and theoptical modulator is configured to direct the plurality of imagesdisplayed by the display panel to different directions.